This disclosure relates generally to storage management in a data processing system, and more specifically, to logical buffer pool extension in a data processing system.
In selected applications where flash memory provides better performance and/or lower cost, flash memory is being used in place of traditional storage mediums (e.g., disks). Accordingly, exploitation of flash memory technology in database systems, for the purpose of providing better performance at lower cost, has become very important. Existing approaches to exploiting the use of flash memory technology in databases systems include using flash storage transparently as a faster disk/tablespace and direct addressing of flash storage from a database buffer pool.
Existing approaches to exploiting the use of flash memory typically have disadvantages including, in one example, operation at a tablespace level, which accordingly has coarse granularity. Entire objects (e.g., tables) must be placed in the flash storage. Individual pages, which are hotter than other pages (i.e., they are more frequently accessed), cannot be specifically placed in a flash memory cache. This approach also typically requires explicit administrative attention to manage and optimize data placement, using empirical data. Explicit administrative attention however does not leverage internal knowledge of the frequency of access characteristics of different data in the database, which is typically known to the database manager. In another example internal access frequency information is used to automatically place hot data into flash storage. However, current approaches typically suffer from other significant disadvantages. In particular, persistence of the content of the flash memory is not maintained, thus resulting in loss of content over system shutdowns and more importantly, no ability to place dirty (updated) data in the flash memory cache.
In another example, rather than treating disk and flash storage as alternative storage options at a same level of a storage hierarchy, flash storage and disk storage are treated hierarchically. All data is initially stored on the disk, and over time, high-value data is identified and stored in the flash storage. This high-value data is kept current on both the flash storage and the disk drive. Other low-value data is current only on the disk, and not resident on the flash storage. Thus, the flash storage behaves like a write-through cache. As with traditional caches, one needs to define suitable admission and replacement policies to optimize performance.
In another example using both software and hardware flash storage caching, a write through caching approach is implemented. In software flash storage caching, a data structure that maps database disk pages to flash storage pages resides in memory. Since memory is volatile, these data structures are lost when power is removed or the system fails leading to data loss. Second, there is concern about the flash storage endurance caused by cell wear. A write-through reduces writes to the flash storage cache, ensuring a longer life span.
In another example, proposed advanced file systems purport to automatically use flash storage for both writes and reads using specific flash storage pools. The proposed file systems can determine data access patterns and store frequently accessed data in a flash-based, level-two read cache. However for sequential writes, the proposed file system utilizes an inherent burst bandwidth of a hard disk. The proposed file systems further constrain use of flash storage by generally restricting physical access to flash storage to a single host system, unless the flash storage is being shared via an exported network file system managed volume. The proposed advanced files systems suggest flash storage is therefore better suited for use in a hybrid storage pool combination of flash storage and traditional hard disk drives.